AFRL-HE-WP-TR-1 999-01 96 


UNITED  STATES  AIR  FORCE 
RESEARCH  LABORATORY 


Preliminary  Recommendation  for  the 
Electronic  Display  of  Graphical  Aircraft 
Maintenance  Information 


Laurie  Quill 
David  Kancier 

University  of  Dayton  Research  Institute 
300  College  Park 
Dayton,  Ohio  45469 


Patrick  Pohle 
Barbara  L.  Masquelier 
Air  Force  Research  Laboratory 


February  1999 


interim  Report  for  the  Period  November  1995  to  February  1999 


2000021/  088 


Approved  for  public  release;  distribution  is  unlimited. 


Human  Effectiveness  Directorate 
Deployment  and  Sustainment  Division 
Logistics  Readiness  Branch 
2698  G  Street 

Wright-Patterson  AFB  OH  45433-7604 


J23&C' ♦A'  1 


NOTICES 


When  US  Government  drawings,  specifications,  or  other  data  are  used  for  any  purpose  other  than 
a  definitely  related  Government  procurement  operation,  the  Government  thereby  incurs  no 
responsibility  nor  any  obligation  whatsoever,  and  the  fact  that  the  Government  may  have 
formulated,  furnished,  or  in  any  way  supplied  the  said  drawings,  specifications,  or  other  data,  is 
not  to  be  regarded  by  implication  or  otherwise,  as  in  any  manner,  licensing  the  holder  or  any 
other  person  or  corporation,  or  conveying  any  rights  or  permission  to  manufacture,  use  or  sell  any 
patented  invention  that  may  in  any  way  be  related  thereto. 

Please  do  not  request  copies  of  this  report  from  the  Air  Force  Research  Laboratory.  Additional 
copies  may  be  purchased  from: 

National  Technical  Information  Service 
5245  Port  Royal  Road 
Springfield,  VA  22161 

Federal  Government  agencies  and  their  contractors  registered  with  Defense  Technical 
Information  Center  should  direct  requests  for  copies  of  this  report  to: 

Defense  Technical  Information  Center 
8725  John  J.  Kingman  Rd.,  STE  0944 
Ft  Belvoir  VA  22060-6218 

TECHNICAL  REVIEW  AND  APPROVAL 

AFRL-HE-WP-TR- 1 999-  0196 

This  report  has  been  reviewed  by  the  Office  of  Public  Affairs  (PA)  and  is  releasable  to  the 
National  Technical  Information  Service  (NTIS).  At  NTIS  it  will  be  available  to  the  general 
public,  including  foreign  nations. 

This  technical  report  has  been  reviewed  and  is  approved  for  publication. 


FOR  THE  COMMANDER 


.TORIGIANylt  Col,  l/SAF 
Deputy  Chief,  Deployment  and  Sustainment  Division 
Air  Force  Research  Laboratory 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  informetion  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  end  maintaining  the  date  needed,  end  completing  end  reviewing  the  collection  of  informetion.  Send  comments  regarding  this  burden  estimate  or  any  ether  aspect  of  this 
collection  of  information,  including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  end  Reports,  1215  Jefferson 
Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  10704-0188),  Washington,  DC  20603. 


3.  REPORT  TYPE  AND  DATES  COVERED 


AGENCY  USE  ONLY  (Leave  blank) 


4.  TITLE  AND  SUBTITLE 

Preliminary  Recommendations  for  the  Electronic  Display  of  Graphical  Aircraft 
Maintenance  Information 


6.  AUTHOR(S) 

Laurie  L.  Quill,  David  E.  Kancler,  Patrick  Pohle,  Barbara  L.  Masquelier 


Interim  -  November  1995  to  Feb 


6.  FUNDING  NUMBERS 

C:  F41624-98-F-5017AB 
PE:  62205F 
PR:  1710 
TA:  DO 
WU:  11 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 
University  of  Dayton  Research  Institute 
300  College  Park 
Dayton,  OH  45469 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

AFRL-HE-WP-TR-1999-0 196 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 
Air  Force  Research  Laboratory,  Human  Effectiveness  Directorate 
Deployment  and  Sustainment  Division 
Air  Force  Materiel  Command 
Logistics  Readiness  Branch 


12a.  DISTRIBUTION  AVAILABILITY  STATEMENT 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 


12b.  DISTRIBUTION  CODE 


Approved  for  public  release;  distribution  is  unlimited. 


13.  ABSTRACT  (Maximum  200  words) 

The  conversion  of  paper  schematic  diagrams  to  electronic  display  presentations  requires  identification  and  analysis  of 
associated  cognitive  demands.  Schematic  diagrams  are  typically  used  by  expert  maintainers  in  troubleshooting  aircraft  faults. 
These  expert  maintainers  must  rely  on  doll,  rule,  and  knowledge-based  behavior  to  successfully  use  these  diagrams.  In  this 
task,  the  Applied  Cognitive  Task  Analysis  (ACTA)  method  was  employed  to  elicit  knowledge  associated  with  using 
schematic  diagrams  for  troubleshooting.  Eleven  F-15  maintenance  technicians  were  interviewed.  Results  showed  that 
schematic  diagrams  not  only  support  the  basic  abilities  required  for  troubleshooting;  they  also  allow  for  visualization  of  the 
dynamic  flow  of  system  relations  and  process  activities  on  the  aircraft.  The  ACTA  method  identified  the  cues  and  strategies 
used  to  mentally  depict  system  flow.  Efforts  to  convert  schematic  diagrams  to  electronic  display  presentations  should 
support  the  basic  troubleshooting  abilities,  as  well  as  the  cues  and  strategies  that  depict  dynamic  system  flow. 


14.  SUBJECT  TERMS 
Schematic  diagram 
Troubleshooting 
Diagnostics 


17.  SECURITY  CLASSIFICATION 
OF  REPORT 

UNCLASSIFIED 


Fault  isolation 


18.  SECURITY  CLASSIFICATION 
OF  THIS  PAGE 

UNCLASSIFIED 


19.  SECURITY  CLASSIFICATION 
OF  ABSTRACT 


15.  NUMBER  OF  PAGES 

18 

16.  PRICE  CODE 


UNCLASSIFIED 


ISSBStfm&WN-  2*89)  1EG) 

D— ignod  uwng  Perform  Pro,  WHS/DIOR,  Oct  84 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


ii 


PREFACE 


The  research  documented  in  this  technical  paper  was  sponsored  by  the  Air  Force 
Research  Laboratory,  Human  Effectiveness  Directorate,  Deployment  &  Sustainment 
Division,  Logistics  Readiness  Branch.  Work  was  performed  by  the  University  of  Dayton 
Research  Institute  (Contract  Number  SP0900-94-D-0001),  300  College  Park,  Dayton 
Ohio,  45469.  Work  was  accomplished  during  the  period  of  Nov  95  through  Jan  97.  The 
program  manager  for  this  contract  was  Mrs.  Laurie  Quill  (UDRI  Human  Factors  Group). 

Authors  would  like  to  acknowledge  the  372  TRS  Det  4  and  Tyndall  AFB,  FL  and 
the  653  CLSS  at  WRALC  for  their  subject  matter  expertise  on  this  program. 


iii 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


iv 


TABLE  OF  CONTENTS 


INTRODUCTION . 1 

Background . 

Maintenance  Troubleshooting  Process . 

Schematics  to  Assist  in  Maintenance . 

Applied  Cognitive  Task  Analysis  in  Determining  Requirements  for  Schematic 

Diagrams . 3 

Current  Study . 3 

METHOD . 4 

Subjects . 5 

Materials . 5 

Procedure . 5 

RESULTS . 6 

Cognitive  Activities  Associated  with  Schematic  Diagrams . 6 

Specific  Examples  of  System  Flow . 7 

Example  1 . 7 

Example  2 . 7 

DISCUSSION . 7 

CONCLUSIONS  AND  RECOMMENTATIONS . 8 

Recommendations . 8 


REFERENCES 


11 


N>  K>  K> 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


vi 


INTRODUCTION 


The  U.S.  Air  Force  has  conducted  research  to  enhance  maintenance  performance  on  the 
flightline,  in  the  depot,  during  peacetime,  wartime,  and  deployment  environments.  A 
primary  focus  of  this  research  has  been  to  determine  the  feasibility  and  capability  of 
presenting,  on  a  compact  display  device,  all  types  of  maintenance  information  required  at 
the  job  site.  The  display  devices  of  choice  include  notebook  computer  displays,  personal 
hand-held  displays,  and  head-mounted  display  devices.  The  types  of  information  required 
for  display  on  these  devices  include  troubleshooting  manuals,  job  guides,  wiring 
diagrams,  and  complex  engineering  schematics.  Complex  engineering  schematics  are 
currently  very  difficult  to  display  on  small  screens.  An  obvious  challenge  is  fitting  these 
large,  complex  schematics  on  small  computer  screens — some  of  the  schematic  diagrams 
fold  out  in  paper  to  at  least  11”  x  17”.  However,  secondary  to  this  issue  is  the 
maintenance  technicians’  use  of  schematic  information.  Technicians  do  not  use  all  the 
information  on  a  schematic  at  the  same  time.  Some  of  the  information  on  the  diagram 
relates  specifically  to  the  maintenance  action  being  performed,  while  other  information 
may  be  relevant  to  the  system  but  not  to  the  current  maintenance  action.  Experienced 
technicians  can  readily  determine  the  relevancy  of  such  information;  however,  novice 
technicians  have  greater  difficulty  with  such  a  cognitive  process.  In  order  for  designers  to 
identify  the  information  requirements  associated  with  schematic  diagram  use,  three  basic 
issues  must  be  addressed.  These  issues  are  as  follows: 

1 )  A  clear  understanding  of  the  maintenance  troubleshooting  process  must  exist; 

2)  An  understanding  of  how  schematics  support  this  process  must  exist;  and 

3)  There  must  be  a  determination  of  what  differences  exist  between  the  expert 
and  novice  maintainer’s  cognitive  processes  and  technical  data  use. 

As  technology  is  fielded,  its  efficient  and  effective  application  must  be  investigated  and 
tested.  But  while  many  potential  problems  can  be  solved  from  the  technological 
perspective  of  hardware  or  software,  the  basis  for  any  solution  must  be  founded  in  the 
needs  of  the  user.  For  example,  in  the  display  of  large  complex  schematics  and  diagrams, 
a  great  deal  of  work  has  been  accomplished  in  developing  methods  for  storing  and 
presenting  technical  data.  However,  additional  research  is  necessary  to  more  precisely 
identify  the  actual  schematic  information  used  by  maintenance  technicians  (use  on 
schematic  diagrams)  to  accomplish  their  tasks. 

The  user  requirements  obtained  in  the  present  study  were  gathered  via  a  cognitive  task 
analysis  (CTA).  This  technique  utilized  knowledge  provided  by  “expert”  maintenance 
technicians.  The  technicians’  knowledge  involved  their  training,  task  performance,  and 
strategies  involving  use  of  schematics  based  on  lessons  learned.  Primary  results  indicate 
the  following:  (1)  the  need  to  maintain  a  global  context,  including  references  to  the 
beginning  and  end  points,  of  particular  path(s)  the  technician  is  following;  (2)  the  need  to 
identify  and  mark  the  path(s)  being  traced;  and  (3)  the  need  to  reduce  the  density  of 
information  presented  simultaneously.  The  technology  necessary  to  target  these  needs 
exists  in  many  forms;  however,  designers  should  design  for  the  needs  of  the  user  rather 
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than  for  the  sake  of  technological  capabilities.  Thus,  acceptable  usability  of  electronic 
schematic  diagrams  will  be  ensured. 


Background 

Maintenance  Troubleshooting  Process 

In  a  review  and  evaluation  of  maintenance  troubleshooting  research,  Morris  and  Rouse 
(14)  identify  three  basic  abilities  used  in  maintenance  troubleshooting.  Consistent  with 
Air  Force  troubleshooting  methods,  these  three  abilities  are  as  follows: 

1)  Replace  components; 

2)  Make  tests  to  eliminate  components  from  considerations;  and 

3)  Search  for  the  problem  in  a  systematic  way  (i.e.,  split  half  or  bracketing 
approach). 

In  their  review,  Morris  and  Rouse  (14)  identify  the  need  for  explicit  instructions  in 
approaching  the  problem,  as  well  as  the  use  of  system  knowledge  in  deciding  appropriate 
actions.  Morris  and  Rouse  also  specify  three  abilities  which  assist  the  subject:  (1)  system 
knowledge;  (2)  the  use  of  algorithms  as  decision  aides;  and  (3)  the  use  of  heuristics. 

Schematics  to  Assist  in  Maintenance 

In  order  to  understand  the  applicability  and  use  of  schematic  diagrams  during  aircraft 
maintenance,  a  basic  understanding  must  exist  of  the  other  types  of  manuals  used  to 
support  maintenance.  Such  Air  Force  maintenance  manuals  include:  the  General  Vehicle 
(GV),  Fault  Isolation  (FI),  Job  Guide  (JG),  Wiring  Diagrams  (WD),  and  Schematic 
Diagrams  (SD).  The  GV  is  used  for  information  about  the  functional  requirements  of  a 
system— to  gain  system  knowledge.  The  FI  guides  the  technician,  logically,  through  the 
troubleshooting  process  to  help  in  diagnosing  a  problem.  The  JG  guides  the  technician, 
step-by-step,  through  specific  maintenance  tasks  such  as  component  replacements  and 
system  checkouts.  Wiring  Diagrams  are  used  to  identify  wiring  connectivity  throughout 
the  airplane  wire  repairs.  The  SD  identifies  functional  connectivity  between  and  within 
systems.  Of  these  manuals,  the  SD  seems  to  require  the  most  cognition  to  use  effectively. 
This  is  evidenced  in  daily  flightline  maintenance:  when  faced  with  a  challenging 
troubleshooting  problem,  an  expert  will  invariably  select  the  SD  to  support  the 
maintenance  activity. 

The  conversion  of  paper  manuals  to  electronic  format  has  necessitated  a  change  in  the 
storage  and  presentation  of  maintenance  technical  data.  For  example,  FI  manuals  have 
been  designed,  electronically,  as  expert  systems  using  Baesian  logic  (2,  3,4).  In  the  case 
of  the  F-16,  these  manuals  exist  as  logic  trees.  Job  Guides  have  been  scanned  into  file 
formats  such  as  PDF  and  have  been  authored  into  databases  which  permit  increased 
interactivity  and  use  of  the  data— Interactive  Electronic  Technical  Manuals  (IETMs)  (5, 
6,  7).  Wiring  Diagrams  have  been  improved  so  that  they  can  be  readily  displayed  on 
small  screens,  as  exemplified  by  several  different  research  efforts  (11,  17).  The 
conversion  of  paper  schematic  diagrams  to  electronic  data,  however,  has  been  less 
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straightforward.  This  is,  perhaps,  due  to  the  cognitive  requirements  associated  with  using 
these  diagrams  (15). 

Applied  Cognitive  Task  Analysis  in  Determining  Requirements  for  Schematic 
Diagrams 

Since  the  late  1980s,  several  methods  have  been  developed  for  eliciting  the  knowledge 
associated  with  expert  decision-making  (8,  9,  12).  The  purpose  of  these  methods  is  to 
identify  the  cognitive  aspects  associated  with  performing  tasks  at  an  expert  level.  These 
methods  have  provided  valuable  insight  into  the  cognitive  aspects  associated  with  the 
troubleshooting  process  of  aircraft  maintenance,  and  they  have  provided  usable  and  useful 
means  of  gathering  cognitive-specific  information  from  experts. 

The  Precursor,  Action,  Result  and  Interpretation  (PARI)  method  provides  for  the 
cognitive  components  of  complex  problem  solving  tasks  (8).  The  method  was  developed 
and  used  to  augment  existing  task  analysis  techniques  by  capturing  both  behavioral  and 
cognitive  components  of  troubleshooting.  The  current  study  emphasizes  the  importance 
of  real  world  learning  in  development  of  expert  knowledge.  The  study  was  in  support  of 
an  integrated  skill  analysis  and  instructional  development  effort.  While  the  study  did  not 
specifically  address  use  of  schematic  diagrams,  the  emphasis  on  training  and  real-world 
learning  in  maintenance  troubleshooting  provided  valuable  perspective  on  the  type  of 
expert  technicians  required  for  the  current  study — instructors. 

The  Applied  Cognitive  Task  Analysis  (ACTA)  method  was  selected  for  collecting 
cognitive  information  for  the  current  study.  This  method  has  been  found  both  useable 
and  useful  in  collecting  knowledge-based  information  from  experts.  Three  techniques  are 
associated  with  ACTA.  The  first  technique  is  the  Task  Diagram  interview.  This 
technique  gives  a  broad  overview  of  the  task  through  a  simple  task  flow  diagram  (e.g., 
three  to  seven  steps  associated  with  the  task).  The  Task  Diagram  is  then  used  to  highlight 
the  complex  decision-making  steps  associated  with  the  task.  The  second  technique,  the 
Knowledge  Audit,  identifies  the  complex  decision-making  steps  through  a  series  of 
questions  or  probes.  The  intent  of  these  probes  is  to  differentiate  the  expertise  associated 
with  the  knowledge-based  decisions  and  those  that  are  rule-based.  The  use  of  cues  and 
strategies  is  emphasized  as  a  method  to  identify  differences  between  a  novice’s  approach 
to  the  problem  and  an  expert’s  approach.  The  third  technique,  the  Simulation  Interview, 
puts  data  collection  within  the  context  of  a  specific  task  scenario,  thereby  allowing  the 
expert  to  give  real-world  examples  of  the  expert  decision  making  cues  and  strategies. 
Each  of  these  three  techniques  was  used  in  the  current  study. 

Current  Study 

The  overall  objective  of  this  effort  was  to  identify  and  evaluate  the  types  and  levels  of 
information  detail  used  or  needed  by  the  technician  in  order  to  successfully  perform  tasks 
that  require  schematic  diagrams.  Upon  defining  the  actual  types  and  quantities  of 
required  pictorial  information,  appropriate  display  format  strategies  were  investigated. 
The  objective  was  not  to  merely  address  the  display  of  large  complex  graphics  on  small 
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display  devices,  but  to  also  identify  the  graphic  support  required  to  ensure  successful  task 
completion.  The  final  requirement  was  to  best  meet  the  above  needs  in  a  flightline 
maintenance  environment. 

The  current  study  drew  subjects  from  the  F-15  flightline  training  facility  at  Tyndall  AFB 
and  Wamer-Robins  ALC  to  collect  cognitive  components  associated  with  using 
schematics  for  troubleshooting  tasks.  Participants  at  Tyndall  included  a  total  of  six 
maintained,  all  instructors.  Participants  at  Wamer-Robins  ALC  included  three 
maintainers. 


METHOD 

Using  the  criteria  of  Hall  et  al.  (8),  the  effort  began  with  the  identification  of  expert 
technicians.  The  current  study  recruited  F-15  instructors  for  initial  data  collection.  These 
instructors  were  chosen  for  their  ability  to  provide  a  balanced  emphasis  on  training 
strategies  and  knowledge  obtained  through  experience  in  the  operational  environment. 
As  instructors,  these  individuals  were  also  identified  as  capable  to  identify  and 
communicate  the  differences  between  novice  and  expert  technicians.  Because  typical 
training  techniques  are  scenario-based,  selection  of  example  tasks  for  use  during  the 
Simulation  Interviews  was  accomplished  during  subject  screening.  The  only  requirement 
was  that  the  chosen  sample  task  required  the  use  of  a  large,  complex  SD. 

Subjects  were  not  required  to  perform  the  actual  task;  rather,  the  three  techniques 
associated  with  the  ACTA  method  were  employed  to  elicit  information  about  the  task. 
As  per  standard  ACTA  methodology,  Task  Diagrams  were  used  to  break  down  the  overall 
task  into  approximately  six  or  seven  steps.  Procedure-based  steps  were  then  identified 
and  separated  from  those  based  upon  expert  decision-making  processes.  Subject  matter 
experts  were  then  probed,  during  the  Knowledge  Audit  phase,  about  the  knowledge-based 
steps  to  discover  cues  and  strategies  used  to  perform  these  complex  tasks.  Special 
emphasis  was  placed  on  identifying  the  cues  and  strategies  not  apparent  to  the  novice 
technician.  Overall,  the  interview  focused  on  the  information  necessary  to  fulfill  the 
technicians’  decision  requirements  and  improve  performance  in  an  electronic 
environment. 

Two  data  collection  trips  occurred  during  this  study.  The  first  trip  was  to  Tyndall  AFB, 
FL  to  collect  information  from  experienced  flightline  maintainers.  The  second  trip  was  to 
Wamer-Robins  ALC  (WRALC),  GA.  The  second  trip  comprised  three  goals:  (1)  to 
collect  information  specific  to  depot  maintenance;  (2)  to  answer  or  clarify  questions 
generated  after  the  first  data  collection  trip;  and  (3)  to  validate  information  collected  from 
the  first  trip. 
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Subjects 


To  take  advantage  of  the  skills  and  knowledge  available  by  experts,  a  total  of  twelve 
maintenance  technicians  participated  in  the  ACTA.  Eight  instructors  participated  in  the 
first  data  collection  activity  at  Tyndall  AFB,  and  three  technicians  participated  in  the 
second  activity  at  WRALC. 

F-15  instructors  were  from  the  372  TRS  Det  4  at  Tyndall  AFB,  FL.  Their  specialties 
were  as  follows:  six  avionics  specialists,  one  environmental  control  specialist,  and  one 
flight  controls  specialist.  Subjects  participated  at  the  request  of  their  supervisor. 

Three  maintenance  technicians  were  from  the  653  CLSS  at  WRALC.  Two  were  avionics 
specialists;  one  each  on  the  C-130,  and  C-141,  respectively.  The  remaining  subject  was 
an  environmental  specialist.  Again,  subjects  participated  at  the  request  of  their 
supervisor. 


Materials 

ACTA  materials  used  by  the  experimenters  included  data  collection  forms,  a  reference 
list  of  knowledge  audit  probes,  and  audio/video  recording  equipment.  Materials  supplied 
on-site  included  paper  technical  manuals  (i.e..  Fault  Isolation  manuals.  Schematics 
Diagrams,  and  General  Vehicle  manuals),  chalk  boards  (for  drawing  task  diagrams),  and 
large,  transparent  plastic  boards.  Maintainers  commonly  place  these  plastic  boards  over 
Schematic  Diagrams  and  use  grease  pencils  to  trace  flows,  such  as  wire  circuit  signals, 
over  the  schematics. 


Procedure 

At  Tyndall,  two  teams  were  established  for  data  collection.  Two  experimenters 
comprised  each  team.  Within  each  team,  one  experimenter  was  responsible  for  the 
primary  dialog  with  the  subject,  while  the  other  experimenter  was  primarily  responsible 
for  data  documentation.  Each  team  interviewed  four  instructors.  Each  instructor  was 
interviewed  for  approximately  two  hours. 

At  WRALC,  one  team  collected  all  of  the  data  and  there  were  two  experimenters  on  the 
team.  Again,  one  experimenter  was  responsible  for  interacting  with  the  subject,  while  the 
other  experimenter  recorded  information.  Interviews  lasted  for  approximately  two  hours 
each. 

Upon  arrival,  each  subject  was  given  a  brief  description  of  the  data  collection  activity. 
Data  collection  began  with  a  Simulation  Interview.  These  technicians  were  instructors 
and  experienced  technicians;  therefore,  they  were  asked  to  provide  a  troubleshooting 
example  as  the  simulation.  These  scenarios  are  typically  used  for  training  purposes  and 
subjects  had  no  difficulty  providing  simulated  problems.  Subjects  were  asked  to  provide 
simulations  that  required  use  of  schematics.  Following  selection  of  the  simulation,  the 
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Task  Diagram  interview  was  conducted.  This  provided  a  broad  overview  of  the  task  and 
differentiated  between  portions  of  the  task  that  were  knowledge  (cognitively)  oriented 
versus  those  that  were  skill  oriented  (procedural).  Following  the  Task  Diagram 
interview,  a  Knowledge  Audit  was  conducted  for  the  cognitively  oriented  Task  Diagram 
tasks  or  subtasks.  Knowledge  Audit  probes  were  used  to  enhance  this  data  collection 
activity.  The  Knowledge  Audit  comprised  the  majority  of  the  interview.  Supplemental 
materials  were  used  throughout  data  collection  activities  (e.g.,  technical  manuals),  and 
audio  tape  recordings  of  the  sessions  were  made.  Following  both  data  collection 
activities,  experimenters  convened  to  compile  and  analyze  information  gathered  from  the 
ACTAs. 


RESULTS 

Results  from  this  effort  not  only  validate  information  gathered  in  previous  studies  and 
reviews;  they  also  provide  valuable  information  concerning  the  cognitive  activities 
associated  with  using  schematic  diagrams. 

Morris  and  Rouse  (14)  reviewed  the  procedure  of  basic  troubleshooting  activities.  They 
also  identified  differences  between  expert  and  novice  approaches  to  troubleshooting.  Air 
Force  troubleshooting  technical  manuals  have  been  designed  to  support  the  abilities 
identified  by  Morris  and  Rouse:  1)  replace  components;  2)  make  tests;  and  3)  search  the 
problem  in  a  systematic  way. 

Certain  approaches,  like  PARI,  have  addressed  cognitive  components  of  troubleshooting; 
however,  SCOPE  specifically  targeted  cognitive  use  associated  with  schematic  diagrams. 

Cognitive  Activities  Associated  with  Schematic  Diagrams 

While  typical  troubleshooting  activities  must  include  replacement  of  components,  testing, 
and  systematic  searching,  these  activities  tend  to  be  static,  step-by-step  procedures.  For 
example,  a  Fault  Isolation  manual  (FI)  typically  will  recommend  a  systematic  approach  to 
testing  (e.g.,  split  half  approach  based  on  a  given  symptom).  A  test  is  recommended  and, 
based  on  the  test  results,  a  component  repair  plan  is  implemented.  This  approach 
incorporates  all  of  the  abilities  identified  by  Morris  and  Rouse  (14).  Interestingly,  use  of 
Schematic  Diagrams  is  recommended  in  the  FI  for  situations  that  could  not  be  anticipated 
when  the  FI  was  developed.  It  may  be  that  the  SD  aids  the  novice  in  envisioning  a 
dynamic  system  state. 

As  a  result  of  the  ACTA  data  collection,  a  distinct  difference  is  noted  between 
troubleshooting  with  the  FI  and  troubleshooting  with  SD.  Experts  do  not  view  their 
systems  as  static  components  with  checkpoints  between  components.  The  SD  allow 
experienced  technicians  to  envision  the  dynamic  flow  within  and  between  systems  on  the 
aircraft.  These  experts  envision  the  flow  while  using  the  three  abilities  to  isolate  the  fault 
or  system  problem.  While  the  troubleshooting  T.O.  can  be  used  by  experienced 
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technicians  to  envision  this  dynamic  flow,  the  Schematic  diagram  better  depicts  a 
dynamic  flow  of  system  activity.  Furthermore,  each  experienced  technician  was  able  to 
identify  the  cues  and  strategies  they  use  to  mentally  depict  system  flow. 

Specific  Examples  of  System  Flow 

This  report  uses  the  term  “system  flow”  generically.  In  an  electrical  system,  system  flow 
refers  to  an  electrical  signal  sent  through  a  circuit.  In  a  mechanical  system,  flow  refers  to 
mechanical  interrelationships,  such  as  force  and  distance,  which  cause  some  mechanism 
to  move.  In  an  Environmental  Control  System,  flow  may  refer  to  airflow  through 
ductwork  or  plumbing.  The  common  thread  between  these  definitions  of  system  flow  is  a 
directional  movement.  This  movement  begins  at  a  given  location,  follows  a  path  or  series 
of  paths,  and  terminates  at  another  location.  There  are,  generally,  many  locations  along 
the  path  where  movement  can  be  altered.  Depending  on  the  system,  an  “altered” 
movement  may  involve  a  change  in  direction,  a  modification  to  flow  characteristics  (e.g., 
transformation  of  signal),  a  change  in  intensity,  or  a  change  in  type  of  flow.  The 
following  paragraphs  provide  specific  cases  in  which  dynamic  system  flows  were 
identified  during  data  collection. 

Example  1 

An  Avionics  expert  provided  an  example  of  system  flow  through  discussion  of  a  logic 
circuit.  In  this  example,  the  technician  emphasized  the  importance  of  examining  an  AIM- 
120  logic  circuit.  First,  he  emphasized  the  need  for  an  “initiate  signal”  as  an  input,  then 
immediately  emphasized  the  importance  of  knowing  the  specific  inputs  required  for  the 
given  output.  In  this  example,  system  flow  was  an  electrical  current  and  the  technician 
followed  the  current  throughout  the  circuit,  systematically  checking  current  and  testing 
continuity  at  various  locations  along  the  path  of  the  circuit.  Throughout  these  checks  and 
tests  his  main  focus  was  on  the  flow  of  current  through  the  circuit. 

Example  2 

An  Environmental  expert  provided  insight  in  system  flow  through  an  example  of  cockpit 
airflow.  This  technician  indicated  that  he  used  the  troubleshooting  T.O.  to  identify 
components  as  a  list  of  what  to  look  for  in  the  SD.  This  technician  also  identified  that  on 
the  F-15  SD,  there  is  descriptive  information  written  inside  boxes  depicting  aircraft 
components.  This  information  is  imperative  in  helping  to  visualize  the  airflow  of  the 
system.  Similar  to  the  avionics  expert,  this  environmental  specialist  envisioned  the  flow 
of  air  from  a  point  of  origin  along  a  path  to  some  output  location.  Systematic  testing 
along  the  path  helped  to  identify  abnormal  changes  in  airflow.  The  SD  helped  to  show 
the  beginning,  path,  end  and  normal  or  expected  airflow  changes. 


DISCUSSION 

Efforts  to  convert  paper  schematic  diagrams  to  electronic  display  presentations  require 
analysis  of  the  cognitive  demands  associated  with  use  of  the  diagrams.  Cognitive 
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activities  include  not  only  the  basic  tenets  associated  with  maintenance  troubleshooting, 
but  also  the  techniques  used  for  knowledge-based  decision  making.  The  three  abilities, 
identified  by  Morris  and  Rouse  (14),  are  clearly  used  by  expert  maintainers.  The  SD 
provides  for  visualization  of  system  flow  from  a  point  of  origin  to  a  point  of  termination 
The  flow  permits  systematic  searching  on  a  limited  set  of  information.  Schematic 
diagram  symbols  like  valves,  sensors,  and  diodes  help  to  visualize  flow  and  flow 
direction.  Along  the  flow  path,  systematic  tests  can  be  conducted  using  standard  split 
half  and  bracketing  approaches.  Based  on  interruption  of  flow  identified  through  making 
tests,  components  can  be  identified  for  replacement. 

This  effort  used  the  ACTA  to  identify  the  three  essential  abilities  used  in  maintenance 
troubleshooting,  as  well  as  the  essential  system  relations  and  processes  required  for 
knowledge-based  use  of  schematic  diagrams.  Identification  of  the  cognitive  requirements 
associated  with  use  of  schematic  diagrams  provides  human  factors  designers  with  the 
information  requirements  associated  with  electronic  presentation  of  schematic  diagrams. 
When  specific  cognitive  requirements  (such  as  determining  the  power  source  or  providing 
a  global  system  perspective)  are  added  to  electronic  schematic  presentations,  information 
needs  of  both  novice  and  expert  maintainers  are  addressed.  The  following  section  details 
the  general  requirements  that  the  CTA  uncovered. 


CONCLUSIONS  AND  RECOMMENDATIONS 

In  developing  computer  graphics,  many  proposals  have  been  suggested  and  effectively 
implemented.  Differing  levels  of  detail  (13),  differing  types  of  information,  and  the  use 
of  transparency  and  blur  (1)  have  all  been  researched. 

For  the  use  of  schematics  for  aircraft  troubleshooting  procedures,  one  of  the  most 
important  cognitive  features  seems  to  be  the  identification  of  signal  flow  through  the 
system.  Furthermore,  novice  technicians  may  or  may  not  possess  the  ability  to  visualize 
such  a  signal  flow.  As  the  effort  increases  to  computerize  aircraft  maintenance 
procedures,  specifically  schematic  and  wiring  diagrams,  the  cognitive  necessity  of 
depiction  of  signal  flow  will  need  to  be  incorporated.  This  will  not  only  provide  expert 
maintainers  with  information  they  use  on  a  regular  basis,  but  also  provide  novice 
maintainers  with  an  added,  expert  strategy  to  troubleshoot  and  repair  aircraft  systems. 

Recommendations 

The  following  is  a  set  of  eight  recommendations  based  upon  the  results  of  the  ACTA 
conducted  at  the  two  maintenance  sites.  It  is  important  to  note  that  these 
recommendations  do  not  specifically  detail  interface  formats  such  as  screen  layouts,  input 
devices,  or  information  encoding  techniques.  Rather,  these  recommendations  are 
strategic  requirements  that  should  be  implemented  based  upon  the  current,  state  of  the  art 
technology  available  at  the  time  the  electronic  maintenance  system  is  designed  and 
developed. 
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1.  Global  context  of  ‘circuit’  or  flow 

A  common  theme  among  all  technicians  interviewed  was  the  necessity  to  maintain  a 
global  perspective  of  the  system.  This  effort  can  prove  cumbersome;  as  mentioned 
earlier,  paper-based  schematic  diagrams  can  fold  out  to  sizes  greater  than  ll”xl7”. 
However,  this  strategy  was  mentioned  by  every  technician  interviewed.  Furthermore,  the 
instructors  at  Tyndall  cited  the  novice’s  lack  of  ability  to  maintain  global  context  as  a 
significant  hindrance  for  those  with  little  experience.  Thus,  the  representation  of  global 
context  is  seen  as  one  of  the  most  important  capabilities  of  an  electronic  depiction  of 
SDs. 

2.  Start  and  end  points  on  same  screen 

Throughout  the  CTA  process,  technicians  repeatedly  described  the  strategy  of  tracing  the 
signal  flow  in  reverse  from  its  endpoint  to  its  starting  point.  The  strategy  of  a 
representing  a  global  circuit  context  should  include  the  representation  of  a  given  circuit’s 
beginning  and  ending  points. 

3.  Flow  visualization  and  direction  of  flow 

Correct  identification  of  signal  flow  is  necessary  for  the  correct  identification  of 
components  and  wires  which  may  need  to  be  tested  or  repaired.  The  misidentification  of 
malfunctioning  parts  can  directly  lead  to  increased  times  to  repair  and  maintenance  costs. 

4.  Changes  in  flow  direction 

At  certain  points  along  the  flow  of  a  signal,  the  direction  or  intensity  of  the  signal  may 
change.  This  is  in  part  due  to  system  state,  but  may  also  be  due  to  a  faulty  system 
component.  Technicians  will  often  use  such  a  disconnect  between  what  should  be  and 
what  actually  is  to  identify  possible  malfunctioning  system  components  (e.g.,  volume 
diodes,  valves,  sensors). 

5.  Provide  system  information  (general  vehicle)  on  supplementary  screen 

Most  of  the  information  used  by  the  technician  from  the  SD  is  pictorial  in  nature. 
However,  there  also  exists  the  need  to  access  detailed  information  about  system 
components  and  wire  bundles.  While  such  information  can  be  referenced  from  the  SD,  it 
is  often  found  in  a  separate  manual  such  as  the  job  guide  or  general  vehicle.  Improving 
the  access  to  such  information  will  help  to  reduce  technician  workload  and  overall  repair 
times. 

6.  General  aids  for  using  split-half  and  bracketing  methods 

Any  electronic  maintenance  aid  should  support  the  task  of  troubleshooting.  While  this 
support  could  be  implemented  many  different  ways,  the  capability  should  be  readily 
available  to  the  technician  via  the  electronic  maintenance  device.  Such  a  capability  will 
help  to  reduce  the  overall  time  and  cost  of  maintenance. 

7.  Symbolize  major  components  on  and  A/C  structures  in  the  circuit 

Current  schematic  diagrams  do  not  depict  the  relation  between  structural  and  functional 
aspects  of  the  aircraft.  However,  maintainers  indicated  that  such  a  depiction  would  be 
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valuable,  especially  when  troubleshooting  a  system  that  is  not  localized  on  the  aircraft, 
(e.g.,  bulkheads). 

8.  Minimize  representation  of  components  and  A/C  structures  not  on  circuit 

As  per  Tufte  (16),  only  essential  information  should  be  highlighted  as  part  of  an 
electronic  display  format.  For  essential  information,  several  graphical  and  text-based 
highlighting  strategies  exist,  such  as  color  coding  and  using  large  or  boldface  text.  Also, 
many  ‘de-highlighting’  strategies,  such  as  graying  out  or  using  smaller  fonts,  can  be 
employed  to  shift  emphasis  away  from  non-essential  information. 
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